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(54) Buffer system for controlled and synchronised presentation of MPEG-2 data services 



(57) A data elementary buffer (58) with underflow 
and overflow operational constraints is incorporated into 
a digital television receiver to regulate the flow of data 
related to an ancillary data service to be presented in 
synchronization with a video or audio program element. 
The data elementary buffer (58) ensures that data is 
received in time for decoding and presentation in syn- 
chronization with the video or audio element. The data 



elementary buffer (58) also limits the amount of data 
that the receiver may be required to cache. The mini- 
mum size of the data elementary buffer (58) is three 
times the size of a nominal data access unit or three 
times the quantity of data that the receiver can receive 
at the maximum rate in the period that a video element 
is displayed by the receiver. 
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Description 

BACKGROUND OF THE INVENTION 

5 [0001] The present invention relates to digital television (DTV) and, more particularly, to a buffer system for a DTV 
receiver useful in presenting broadcast data services synchronized to the audio or visual content of a digital television 
program. 

[0002] A digital television system includes a transmitter and a receiver which assemble programs comprising audio, 
video, and ancillary components for coordinated presentation to a user. The transmitter system includes subsystems to 

10 receive and compress the digital source data (the elementary or application data streams representing a program's 
audio, video, and ancillary data components); multiplex the data from the several elementary data streams into a single 
transport bit stream; and transmit the data to the receiver. At the receiver the transport bit stream is demultiplexed into 
its constituent elementary data streams. The elementary data streams are decoded and the audio and video data 
streams are delivered as synchronized program elements to the receivers presentation subsystem for display as parts 

is of a coordinated television program. The digital television system is described by the ATSC DIGITAL TELEVISION 
STANDARD, Advanced Television Systems Committee, Doc A/53, 23 Dec 97 incorporated by reference herein. The 
system employs the MPEG-2 systems and video stream syntax of ISO/IEC DIS 13818-1 & 2, INFORMATION TECH- 
NOLOGY - GENERIC CODING OF MOVING PICTURES AND ASSOCIATED AUDIO INFORMATION, International 
Standards Organization, 1995, 16 Sep 95, incorporated by reference herein. Audio coding is in accordance with the 

20 DIGITAL AUDIO COMPRESSION STANDARD (AC-3), Advanced Television Systems Committee, Doc A/52, 16 Nov 94, 
20 Dec 95, incorporated by reference herein. 

[0003] In addition to the basic audio and video content of a teievison program, digital television provides the opportu- 
nity to augment teievison programming with a range of ancillary data services. Ancillary data services described in the 
ATSC DIGITAL TELEVISION STANDARD include control data; conditional access control data associated with limited 
25 access to scrambled signals; and data associated with program audio and video services such as "closed captioning," 
emergency broadcasts, and a program guide containing information about the audio, video, and data content of pro- 
grams being transmitted or to be transmitted in the future. 

[0004] In addition to these basic ancillary data services, the digital teievison standard envisions the broadcast of addi- 
tional ancillary or private data services. The contents of these services are to be defined in the future by developers of 

30 digital television content. These data services can include asynchronous, synchronous, and synchronized streaming 
and non-streaming data. Synchronized streaming and non-streaming data afford the opportunity to provide data serv- 
ices that are associated with a particular audio or visual instant of a television program. The suite of ATSC digital tele- 
vision standards will include a specification for data broadcasting drafted by the T3S1 3 working group of the Advanced 
Television Systems Committee. This ATSC DATA BROADCAST SPECIFICATION, currently in draft form, is expected to 

35 include data and system specifications to facilitate data transmission in a manner compatible with the digital multiplex 
bit streams of ISO/IEC 13818-1 (MPEG-2 Systems) used in digital television video broadcasting. The specification is 
also expected to describe mechanisms necessary to permit ancillary data to be associated with particular audio or vis- 
ual content in a television broadcast. 

[0005] The ATSC DIGITAL TELEVISION STANDARD describes a model for a hypothetical ancillary services target 
40 system decoder for a DTV receiver. This idealized decoder is modeled after a conceptual model for a transport stream 
system target decoder (T-STD) described in the ISO/IEC 13818-1:1996 (MPEG-2) standard. The MPEG-2 conceptual 
T-STD model comprises an individual system target decoder for each of the elementary data streams representing the 
video, audio, and system information components of a television program. However, the MPEG-2 system specification 
does not define a system target decoder for ancillary, synchronized data services. 
45 [0006] The ATSC model ancillary services target system decoder includes a buffer system linking the receiver's trans- 
port stream demultiplexer with the application decoder for the specific type of elementary data stream being decoded. 
The transport stream demultiplexer separates and reconstructs the several elementary streams from the single multi- 
plexed MPEG-2 transport bit stream. The application decoders decode the data elements or access units making up 
the data elementary streams in preparation for presentation to the user. The ATSC model ancillary services target 
so decoder, as well as the decoder described in the European Digital Video Broadcasting Consortium (DVB) specification, 
are two buffer systems with a 512-byte transport buffer followed by a smoothing buffer. 

[0007] For synchronized presentation of streaming data the DTV receiver must be able to collect the packets of data, 
reassemble them into data access units (an elemental unit of data to be decoded and presented in association with an 
instant of a video or audio program), and send them to the application decoder that is processing the data and convert- 
55 ing the data to a form suitable for presentation at the desired time instant. If the receiver receives the data earlier or at 
a faster rate than required, the excess data must be stored in a buffer. On the other hand, sufficient data must be readily 
available in the receiver or the presentation will not be smooth and properly synchronized to the video or audio content. 
[0008] The ATSC model for an ancillary service target decoder consists of a transport demultiplexing buffer and a 
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smoothing buffer. However, there is no buffer for reconstructing data access units to be decoded and then presented in 
synchronization with the audio or visual elements of a program. Likewise, the size of a data access unit, the frequency 
of transmission of data access units, and the method of synchronization is not established by the ATSC DIGITAL TEL- 
EVISION STANDARD and not contemplated by the MPEG-2 specification. Derivation of the general T-STD model for 

5 all kinds of ancillary data is not straightforward because the decoding model for ancillary data can be anything from sim- 
ple to complex. As a result, the quantity of data the receiver may be required to cache might exceed its capacity (a con- 
dition of memory overflow) or throughput capabilities. Alternatively, the receiver may fail to receive all of the data in a 
data access unit in time for decoding and synchronized presentation with a specified instant in the audio or video data 
streams resulting in a loss of data and inconsistent performance (a condition of memory underflow). 

10 [0009] What is desired, therefore, is a buffer system for an ancillary service target decoder for synchronized data serv- 
ices that establishes the size limits of a data access unit for those services and enforces a transmission schedule on 
the MPEG-2 multiplexer of the transmitter or emission station such that the data arrives in time for proper synchronized 
presentation but in a quantity and at a rate which will not overwhelm the receiver so no data is lost. 

15 SUMMARY OF THE INVENTION 

[001 0] The present invention overcomes the aforementioned drawbacks of the prior art by providing a buffer mecha- 
nism to regulate delivery of data for synchronized presentation by a digital television receiver that includes a smoothing 
buffer, the buffer mechanism comprising a data elementary buffer constrained to operate without overflow and under- 
go flow of data, to receive the data from the smoothing buffer. Further, the size of the data elementary buffer is a function 
of the quantity of data that can be received by the receiver in the period in which it displays a video element, such as a 
frame or a field. 

[001 1 ] A complete target system decoder for a synchronized ancillary data service for a digital television receiver is 
specified comprising a transport buffer receiving transport stream data packets for the synchronized ancillary data serv- 

25 ice from a transport stream demultiplexer; a smoothing buffer receiving transport stream data packets from the trans- 
port buffer at a transport buffer leak rate; a first-in, first-out data elementary buffer, constrained to operate without 
overflow and underflow of data, receiving elementary data stream packet payload data from the smoothing buffer at a 
smoothing buffer leak rate and aggregating the elementary data stream packet data to reconstruct a data access unit; 
and an application decoder receiving the data access unit from the data elementary buffer at a specified time. 

30 [0012] Further, a method of buffering data for synchronized presentation with an instant of a visual or audio data 
stream by a digital televison receiver is provided comprising setting an appropriate size for a data elementary buffer to 
aggregate data to reconstruct an access unit of data, the data elementary buffer being constrained to operate without 
overflow or underflow of data; removing an access unit of data from the data elementary buffer for presentation at a 
removal time correlated with a presentation time stamp included in the visual or audio data stream; and adding data to 

35 the data elementary buffer from a smoothing buffer. 

[0013] The buffer of the ancillary service target decoder causes the flow of data from the DTV transmitter to the 
receiver to be regulated so that data to be presented in synchronization with an instant of an audio or visual element of 
the program will be available in the ancillary data services application decoder in time to be decoded and presented in 
coordination with the corresponding instant of the visual or audio program element. The operating constraints of the 

40 data elementary buffer impose conditions that must be observed by the multiplexer system at the emission station to 
prevent overflow of data which would result in the loss of data. The size of the data elementary buffer limits the quantity 
of data that can be held at the receiver and, therefore, the period of time the data can be held before decoding. A min- 
imum time interval between removal of two consecutive data access units provides an upper bound on the data transfer 
capabilities to be supported in data receivers. 

45 [0014] The foregoing and other objectives, features and advantages of the invention will be more readily understood 
upon consideration of the following detailed description of the invention, taken in conjunction with the accompanying 
drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

50 

[0015] 

FIG. 1 is a block diagram of a digital television transmitter and receiver 

FIG. 2 is a block diagram of an ancillary data service target decoder according to the present invention. 

55 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0016] Referring to FIG. 1 , a digital television system comprises a transmitter 2 or emission station and a receiver 4. 
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The transmitter 2 includes facilities to receive a plurality of streams of digital source data 6, each stream representing 
a video, an audio, or an ancillary data services component of a television program. Ancillary data services described in 
the ATSC DIGITAL TELEVISION STANDARD include control data, conditional access data which scrambles the pro- 
gram unless the decoder has the proper key, and data associated with program audio and video services, such as sys- 
5 tern information for acquiring the program elements, information for a program guide, emergency messages, and 
"closed captioning" services. 

[0017] In addition to the basic ancillary data services, the ATSC standard envisions the broadcast of additional ancil- 
lary or private data services. The scope of private data services is expected to be defined in the future as new uses and 
features are developed for the digital television system. For example, such services might include interactivity features 
w permitting the user to respond to the video and audio content of a television program. The ATSC standard envisions 
these data services being transmitted as asynchronous, synchronous, or synchronized streaming and non-streaming 
data. 

[0018] In the transmitter 2 the source data streams 6 are processed in application encoders 8 which perform data 
encoding and formatting required before transmission. Application encoders 8 include data servers for ancillary data 

is elementary streams. The digital television system employs compression techniques from the MPEG-2 toolkit and the 
video stream syntax specified in the MPEG-2 standard, ISO/IEC 13818-1, INFORMATION TECHNOLOGY - GENERIC 
CODING OF MOVING PICTURES AND ASSOCIATED AUDIO INFORMATION, International Standards Organization, 
1995. Audio coding is in accordance with the DIGITAL AUDIO COMPRESSION STANDARD (AC-3), Advanced Televi- 
sion Systems Committee, Doc A/52, 16 Nov 94, 20 Dec 95. The data servers generate data packets according to the 

20 protocol encapsulation, packetization layers, synchronization layers, and protection layers defined in the draft ATSC 
T3S13 DATA BROADCAST SPECIFICATION. The encoded and formatted elementary bit streams 10 are passed to the 
transport packetization and multiplexing subsystem 12 where each of the elementary data streams 10 is divided into 
packets. Input of a program element to the transport packetization and multiplexing unit is in accordance with the limi- 
tations imposed by the transport system target decoder model 31 associated with that program element. For example, 

25 transport packetization and multiplexing of ancillary services information is in accordance with the limitations imposed 
by the ancillary services system target decoder model described herein. This includes conditions to prevent overflow 
and underflow of data and an excessive data transfer rate to the receiver. Timing information is included in the packets 
of the elementary streams and the several packetized elementary streams are multiplexed into a single transport bit 
stream 14. The data encoding and packetization are synchronized by a system clock 18. The transport bit stream 14 is 

30 passed to a modem subsystem 20 for transmission. In terrestrial broadcast television the modem subsystem 20 may 
be a radio frequency transmission system which uses the digital data in the transport stream to modulate a transmitted 
signal 22. Other modem systems could be used for satellite or cable transmission and, at least some, systems are 
expected to include two-way communication for interactivity. 

[0019] The transmitted signal 22 is received by a modem subsystem 27 in the receiver 4. The transport data stream 
35 26 with error signaling is forwarded to the transport depacketization and demultiplexing subsystem 28 where the ele- 
mentary bit streams 30 for each of the program elements are demultiplexed and recovered from the transport data 
stream 26. The buffers 31 of the transport stream system target decoders, which include individual target decoders for 
the video, audio, system data and ancillary data streams, link the transport demultiplexing system 28 with the applica- 
tion decoders 32. In the application decoders 32 streams of consecutive elementary audio, video, and ancillary data 
40 access units are decoded and reassembled into elementary data streams 34. The elementary data streams 34 are for- 
warded to the presentation subsystem 36 for display, sound generation or other action. Decoding and presentation of 
the program elements are coordinated by a system clock control 38 which is derived from the program clock references 
(PCRs) transmitted in the bit stream 1 4. t 

[0020] Referring to FIG. 2, the buffers of the ancillary service target decoder for synchronized data services 40 of the 
45 present invention link the transport stream demultiplexer 42 and the application decoder 43 for a synchronized ancillary 
data service data stream 44. The transport demultiplexer 42 separates the transport bit stream 41 into streams of trans- 
port packets for each of the program elements which can include a data stream for video 46 and one or more streams 
for audio 48, system data 50 and ancillary data services 44 and 52. The transport demultiplexer 42 directs the data in 
each of the transport packet data streams to a transport system target decoder for that stream. In the ancillary data 
so service target decoder for synchronized data services 40, complete transport data packets as indicated by the packet 
identifier (PID) in the transport packet header are passed to the transport buffer 54 at the transport stream rate. Packets 
transferred to the transport buffer 54 include duplicate packets and packets without payloads. The purpose of the trans- 
port buffer is to prevent transmission of large bursts of transport packets pertaining to the same program element. Con- 
sistent with the ISO transport system target decoder model, the size of the transport buffer 54 of the ancillary services 
55 target decoder for synchronized data services 40 is 51 2 bytes. 

[0021] The draft ATSC DATA BROADCAST SPECIFICATION includes the definition of data service profiles and levels 
for DTV systems. These capabilities are signaled in a data broadcast descriptor structure in an event information table 
or data information table of the ATSC program and system information protocol (PSIP). The maximum data rate of a 
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receiver varies according to an MPEG-2 specified data service profile, as follows: 



Data Service Profile 


G1 


G2 


G3 


A1 


Maximum Data Rate 


383,896 bps 


3,838,960 bps 


19.2 Mbps 


19.2 Mbps 



[0022] If there is data in the transport buffer, the data is removed from the transport buffer 54 at a rate of 1 .2 times the 
w maximum data rate of the system as identified by the MPEG-2 data service profile for the broadcast. In the transport 
system target decoder of the present invention, the transport buffer 54 is not permitted to overflow to prevent loss of 
data. This constraint must be taken into account by the data server of the emission station so that no ancillary service 
transport packet is inserted into the multiplexer that could trigger an overflow in the transport buffer 54. 
[0023] Data bytes 55 in the transport buffer 54 that are part of an ISO/IEC 13818-1 packetized elementary stream 
15 (PES) packet or an ISO/IEC 1 381 8-6 digital storage media command and control (DSM-CC) section are transferred to 
a smoothing buffer 56. Other bytes in the transport stream packet and duplicate packets are not included in the data 55 
transferred to the smoothing buffer 56. The smoothing buffer is not permitted to overflow. The size of the smoothing 
buffer may be determined by descriptors in the bit stream 26 but is typically 4096 bytes for the ancillary service target 
decoder 40. 

20 [0024] Ail data bytes 60 of data elementary stream packets of a synchronized data service elementary data stream 
are passed to the data elementary buffer 58 from the output of the smoothing buffer 56. The data elementary buffer 58 
is a first in -first out buffer from which all bytes representing a data access unit of a streaming synchronized data service 
are removed at the time specified by the presentation time stamp (PTS) of the elementary stream packet, assuming that 
decoding of the data access unit is instantaneous. Once decoded, each data access unit is presented at the time spec- 

25 rfied by the presentation time stamp (PTS) value associated with the data access unit. Presentation includes any action 
triggered at the time specified by the PTS, including display on a monitor. Presentation of the data access unit is syn- 
chronized with the audio and video content of the program presented at same PTS instant. 
[0025] Since decoding is not, in reality, instantaneous, a data access unit must be removed from the data elementary 
stream buffer at a time earlier than the time specified for presentation to allow lor decoding. For streaming synchronized 

30 data elementary streams, a data access unit may be removed from the data elementary stream buffer as early as the 
time specified by the PTS for the previous data access unit in effect the DTS of a data access unit, may be considered 
to be equal to the PTS of the previous access unit. For non-streaming synchronized data elementary streams, data 
access units are removed from the data elementary buffer at a time explicitly or implicitly specified for that data access 
unit. A decoding time stamp (DTS) in the bit stream and associated with the data access unit may be used for explicit 

35 specification of decoding time. Implicit specification of decoding time may be accomplished by a global offset signaled 
outside of the data elementary stream that specifies the time before the appropriate PTS for removing the data access 
unit from the data elementary buffer. The decoding time associated with any data access unit is only an indication of 
the actual time required and receivers may take an access unit at a later time if the receiver has sufficient computing 
capability to perform decoding in a lesser amount of time. 

40 [0026] The present inventor realized that a data elementary buffer that is not allowed to overflow would set an upper 
limit on the size of the basic data unit of the elementary data stream (the data access unit). Data loss occurs if the data 
elementary buffer should overflow and, therefore, data is not transferred from the smoothing buffer 56 if the data ele- 
mentary buffer 58 is full. Further, the data elementary buffer imposes a transmission schedule on the multiplexer of the 
transmitter causing time sensitive broadcast data to be delivered to the receiver in a controlled fashion. For example, if 

45 the data elementary buffer is not permitted to underflow, a data access unit must have been received in full before it is 
removed from the buffer. A no underflow constraint in the receiver, therefore, requires that complete data access units 
be delivered to the data decoder 43. In addition, restricting the time between the PTS of two consecutive data access 
units in the data elementary buffer guarantees a maximum data transfer capability (throughput) in the receiver. Such a 
restriction is desirable to establish the speed of the receiver's hardware or the speed of the interfaces forwarding data 

so access units to external devices. The maximum rate of data transfer from the data elementary buffer 58 to the ancillary 
data service application decoder 43 is determined by minimum time that must elapse between the PTS of two consec- 
utive data access units. To take advantage of a variety of potential output devices, including television and computer 
monitors, this time is set at 5.561 1 1 1 milliseconds in the ATSC DATA BROADCAST SPECIFICATION so that the max- 
imum leak rate should not exceed 200 Mbps for an IEEE 1394 data bus. 

55 [0027] The data in the synchronized ancillary data services elementary stream is to be displayed in synchronized 
association with a particular instant in another elementary stream, most likely the video stream. Therefore, the inventor 
concluded that the size of the nominal data access unit and the data elementary buffer 58 should be a function of the 
quantity of data that the system can receive during a reference period of significance to the video data stream. As an 
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example, the video frame rate of the DTV system is 30 frames per second. For the data related to a synchronized data 
service to remain synchronized with the video, the quantity of data in a unit to be decoded for display with a frame of 
video or a data access unit cannot exceed the maximum data rate of the system multiplied by the frame period (1/30th 
sec). For a system capable of a data service maximum bit rate of 19.2 Mbps, the maximum size of a data access unit 

5 synchronized to the frame rate is approximately 80 KBytes (1/30 sec. X 19.2 Mbps). Likewise, nominal data access 
units could be sized for association with some division or multiple of the frame rate. Sizing the data access unit for trans- 
mission at the field rate of the DTV system (60 fields per second) results in a data access unit of 40,040 bytes with a 
nominal frequency of 59.94 Hz at 19.2 Mbps. To handle data at this rate the minimum leak rate required at the output 
of the data elementary buffer 58 is 1 72.8 Mbps. 

10 [0028] The size of the data elementary buffer 58 is derived from the size of a data access unit and the desired function 
of the buffer. The inventor concluded that the minimum size of the data elementary buffer 58 for proper functioning 
should be three nominal data access units. Therefore, the minimum size of the data elementary buffer 58 is 120,120 
bytes. A buffer of this size permits a first data access unit to be in the process of being removed from the buffer, a sec- 
ond data access unit to be held ready, and a third data access unit to be in the process of being input to the buffer. In 

15 the ancillary data services data elementary stream, the size of data access units can vary as long as no overflow or 
underflow of the data elementary buffer occurs. If a transport stream contains multiple synchronized data elementary 
streams and these data streams belong to the same data service, the data elementary buffer is divided equally to 
accommodate the acquisition and reconstruction of the data access units of each stream. 

[0029] If the data elementary buffer is not full, data is transferred from the smoothing buffer 56 to the data elementary 
20 buffer 58 at a leak rate which is function of the maximum data rate of the system for the data services profile. The max- 
imum terrestrial data rate is approximately 19.2 Mbps (allowing for overhead in the transport stream). A smoothing 
buffer leak rate of 1 9.2 Mbps is adequate for systems incorporating the maximum data transmission rate but excessive 
for a system with a G1 service profile with a maximum data rate of 384 Kbps. Additional data capacity can be incorpo- 
rated into the data elementary buffer to accommodate different levels of data services with higher data transfer rates. 
25 [0030] All the references cited herein are incorporated by reference. 

[0031] The terms and expressions that have been employed in the foregoing specification are used as terms of 
description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding equiv- 
alents of the features shown and described or portions thereof it being recognized that the scope of the invention is 
defined and limited only by the claims that follow. 

30 

Claims 

1 . A buffer mechanism to regulate delivery of data for synchronized presentation by a digital television receiver that 
includes a smoothing buffer (56), said buffer mechanism comprising a data elementary buffer (58) constrained to 

35 operate without overflow and underflow of said data, to receive said data from said smoothing buffer (56). 

2. The buffer mechanism of claim 1 wherein a data capacity of said data elementary buffer (58) is a function of a quan- 
tity of said data that can be received by said receiver in a period of display of a video element by said receiver. 

40 3. The buffer mechanism of claim 2 wherein said video element is a video frame. 

4. The buffer mechanism of claim 2 wherein said video element is a video field. 

5. The buffer mechanism of claim 1 wherein a data capacity of said data elementary buffer (58) is, at least, approxi- 
45 mately three times a maximum quantity of said data that can be received by said receiver in a period of display of 

a video element by said receiver. 

6. The buffer mechanism of claim 5 wherein said video element is a video frame. 
50 7. The buffer mechanism of claim 5 wherein said video element is a video field. 

8. A target system decoder for a synchronized ancillary data service for a digital television receiver comprising: 

(a) a transport buffer (54) receiving a first plurality of transport stream data packets for said synchronized ancil- 
55 lary data service from a transport stream demultiplexer; 

(b) a smoothing buffer (56) receiving a second plurality of transport stream data packets from said transport 
buffer at a transport buffer leak rate; 

(c) a first-in, first-out data elementary buffer (58), constrained to operate without overflow and underflow of 
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data, receiving elementary data stream packet data from said smoothing buffer (56) at a smoothing buffer leak 
rate and aggregating said elementary data stream packet data to reconstruct a data access unit; and 
(d) an application decoder (43) receiving said data access unit from said data elementary buffer (58) at a spec- 
ified admission time. 

5 

9. The decoder of claim 8 wherein a data capacity of said data elementary buffer (58) is a function of a quantity of said 
data that can be received by said receiver in a period of display of a video element by said receiver. 

10. The decoder of claim 9 wherein said video element is a video frame. 

10 

1 1 . The decoder of claim 9 wherein said video element is a video field. 

12. The target system decoder of claim 8 wherein a data capacity of said data elementary buffer (58) is, at least, 
approximately three times a maximum quantity of said data that can be received by said receiver in a period of dis- 

75 play of a video element by said receiver. 

13. The target system decoder of claim 12 wherein said video element is a video frame. 

14. The target system decoder of claim 12 wherein said video element is a video field. 

20 

15. The target system decoder of claim 8 wherein said admission time for said application decoder is a time specified 
by a presentation time stamp for another said data access unit removed from said data elementary buffer (58) at 
an earlier time. N 

25 1 6. The target system decoder of claim 8 wherein said admission time for said application decoder (43) is a time spec- 
ified by a decoding time stamp. 

1 7. The target system decoder of claim 8 wherein said admission time for said application decoder (43) is specified as 
a period of time in advance of a presentation time stamp for admission of said data access unit to said application 

30 decoder. 

18. A method of buffering data for synchronized presentation with an instant of at least one of a visual and an audio 
data stream by a digital television receiver comprising: 

(a) setting an appropriate size for a data elementary buffer aggregating data to reconstruct a access unit of 
data, said data elementary buffer (58) constrained to operate without overflow or underflow of data; 

(b) removing said access unit of data from said data elementary buffer (58) for presentation at a removal time 
correlated with a presentation time stamp included in said at least one of a visual and an audio data stream; 
and 

(c) adding said data to said data elementary buffer (58) from a smoothing buffer (56). 

1 9. The method of claim 1 8 wherein said appropriate size for said data elementary buffer (58) is a function of a quantity 
of said data that can be received by said receiver in a period of display of a video element by said receiver. 

45 20. The method of claim 19 wherein said video element is a video frame. 

21 . The method of claim 19 wherein said video element is a video field. 

22. The method of claim 18 wherein said appropriate size of said data elementary buffer (58) is, at least, approximately 
so three times a maximum quantity of said data that can be received by said receiver in a period of display of a video 

element by said receiver. 

23. The method of claim 22 wherein said video element is a video frame. 

55 24. The method of claim 22 wherein said video element is a video field. 

25. The method of claim 18 wherein said data is added to said data elementary buffer (58) at a maximum rate that is a 
function of a maximum data rate appropriate for said receiver. 
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26. The method of claim 1 8 further comprising transferring data to said smoothing buffer from a transport buffer (54). 

27. The method of claim 1 8 wherein said removal time is a time specified by a presentation time stamp for another said 
unit of data removed from said data elementary buffer (58) at an earlier time. 

5 

28. The method of claim 18 wherein said removal time is a time specified by a decoding time stamp. 

29. The method of claim 18 wherein said removal time is determined by specifying a period of time in advance of a 
presentation time stamp for removal of said access unit of data from said data elementary buffer (58). 

10 

30. The method of claim 8 wherein said decoding time stamp is explicitly defined. 

31 . The method of claim 8 wherein said decoding time stamp is implicitly defined. 
75 32. The method of claim 28 wherein said decoding time stamp is explicitly defined. 

33. The method of claim 28 wherein said decoding time stamp is implicitly defined. 
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